FusionRCNN: LiDAR-Camera Fusion for Two-Stage 3D Object Detection

Accurate and reliable perception systems are essential for autonomous driving robotics. To achieve this, 3D object detection with multi-sensors is necessary. Existing detectors have significantly improved accuracy by adopting a two-stage paradigm that relies solely on LiDAR point clouds proposal refinement. However, the sparsity of clouds, particularly faraway points, makes it difficult LiDAR-only refinement module to recognize locate objects accurately. address this issue, we propose novel multi-modality approach called FusionRCNN. This effectively efficiently fuses camera images in Regions Interest (RoI). The FusionRCNN adaptively integrates both sparse geometry information from dense texture unified attention mechanism. Specifically, first utilizes RoIPooling obtain an image set size gets sampling raw points within proposals RoI extraction step. Then, leverages intra-modality self-attention enhance domain-specific features, followed well-designed cross-attention fuse two modalities. fundamentally plug-and-play supports different one-stage methods almost no architectural changes. Extensive experiments KITTI Waymo benchmarks demonstrate our method boosts performances popular detectors. Remarkably, improves strong SECOND baseline 6.14% mAP outperforms competing approaches.